{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# KubeHound 101 - Gremlin and DSL\n",
    "\n",
    "A step by step example of basic Gremlin and KubeHound DSL queries."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting started\n",
    "\n",
    "Connection is being initated directly from the docker using the env vars `GRAPH_NOTEBOOK_HOST` and `GRAPH_NOTEBOOK_PORT`. To overwrite it you can use the magic `%%graph_notebook_config` as detailed below.\n",
    "\n",
    "To connect to a different kubegraph server (hosted somewhere for instance), just run the cell below with your own `host` and `port`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "init_cell": true
   },
   "outputs": [],
   "source": [
    "%%graph_notebook_config\n",
    "{\n",
    "  \"host\": \"kubegraph\",\n",
    "  \"port\": 8182,\n",
    "  \"ssl\": false,\n",
    "  \"gremlin\": {\n",
    "    \"traversal_source\": \"g\",\n",
    "    \"username\": \"\",\n",
    "    \"password\": \"\",\n",
    "    \"message_serializer\": \"graphsonv3\"\n",
    "  }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Setting the visualisation aspect of the graph rendering. **This step is also mandatory.**\n",
    "\n",
    "Now set the appearance customizations for the notebook. You can see a guide on possible options [here](https://github.com/aws/graph-notebook/blob/623d43827f798c33125219e8f45ad1b6e5b29513/src/graph_notebook/notebooks/01-Neptune-Database/02-Visualization/Grouping-and-Appearance-Customization-Gremlin.ipynb#L680)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "init_cell": true
   },
   "outputs": [],
   "source": [
    "%%graph_notebook_vis_options\n",
    "{\n",
    "  \"edges\": {\n",
    "    \"smooth\": {\n",
    "      \"enabled\": true,\n",
    "      \"type\": \"dynamic\"\n",
    "    },\n",
    "    \"arrows\": {\n",
    "      \"to\": {\n",
    "        \"enabled\": true,\n",
    "        \"type\": \"arrow\"\n",
    "      }\n",
    "    }\n",
    "  }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To run a query you need to start with the `%%gremlin` magic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "kh                // traversal source (KubeHound DSL) \n",
    ".V()              // retreive all the vertices\n",
    ".count()          // count the number of results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To show a graph you need to add some option to make the graph more readable `%%gremlin -d class -g critical -le 50 -p inv,oute`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "kh                // traversal source (KubeHound DSL) \n",
    ".V()              // retreive all the vertices\n",
    ".path()           // wrap it with a path type (to show into a graph)\n",
    ".by(elementMap()) // get details for each vertices (properties/values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Raw information in the console tab (download CSV or XSLX). The search go through all the fields in the results.\n",
    "\n",
    "Graph view to navigate through the results (can access properties info through the burger button when a vertice is selected).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Constructing requests\n",
    "\n",
    "Every vertices has a label associated which describes the type of the k8s resources (can be accessed through KubeHound DSL).\n",
    "\n",
    "Raw gremlin query to select all pod in a k8s cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "kh               // traversal source (KubeHound DSL) \n",
    ".V()             // retreive all the vertices\n",
    ".hasLabel(\"Pod\") // retreiving all the pods\n",
    ".valueMap()      // transforming it to json with all properties value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Equivalent in KubeHound DSL:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "kh               // traversal source (KubeHound DSL) \n",
    ".V()             // retreive all the vertices\n",
    ".hasLabel(\"Pod\") // retreiving all the pods\n",
    ".valueMap()      // transforming it to json with all properties value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "List all nodes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "kh           // traversal source (KubeHound DSL) \n",
    ".nodes()      // retreiving all the nodes\n",
    ".valueMap()  // transforming it to json with all properties values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "List all volumes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "kh           // traversal source (KubeHound DSL) \n",
    ".volumes()      // retreiving all the volumes\n",
    ".valueMap()  // transforming it to json with all properties values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "List all endpoints"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "kh           // traversal source (KubeHound DSL) \n",
    ".endpoints()      // retreiving all the endpoints\n",
    ".valueMap()  // transforming it to json with all properties values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "List all containers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "kh           // traversal source (KubeHound DSL) \n",
    ".containers()      // retreiving all the containers\n",
    ".valueMap()  // transforming it to json with all properties values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "List all users:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "kh           // traversal source (KubeHound DSL) \n",
    ".users()      // retreiving all the users\n",
    ".valueMap()  // transforming it to json with all properties values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "List all groups:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "kh           // traversal source (KubeHound DSL) \n",
    ".groups()      // retreiving all the groups\n",
    ".valueMap()  // transforming it to json with all properties values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "List all service accounts:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1e29259d77ba4c85b866d74ca9b10547",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Tab(children=(Output(layout=Layout(max_height='600px', max_width='940px', overflow='scroll')), Output(layout=L…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%gremlin\n",
    "kh           // traversal source (KubeHound DSL) \n",
    ".sas()      // retreiving all the services account\n",
    ".valueMap()  // transforming it to json with all properties values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For each type you can select specific resources based on its name (one or many).\n",
    "\n",
    "Let's select 3 containers with specific names:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "kh           // traversal source (KubeHound DSL) \n",
    "             // selecting multiples containers with specific name\n",
    ".containers(\"nsenter-pod\",\"pod-create-pod\", \"host-read-exploit-pod\")\n",
    ".valueMap()  // transforming it to json with all properties values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For each type you can select specific resources based on its name (one or many). To get the exhaustive list you can use `.properties()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "kh            // traversal source (KubeHound DSL)   \n",
    ".containers() // selecting multiples containers with specific name\n",
    ".limit(1)     // limiting result to 1 container only\n",
    ".properties() // printing the properties and the associated values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most important common properties present for all KH resources."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin\n",
    "kh.containers().limit(1)\n",
    ".properties(\"runID\",\"app\",\"cluster\",\"isNamespaced\", \"namespace\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To select resources with specific properties, use the `.has()` and `.not()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "kh.containers()\n",
    ".has(\"image\",\"ubuntu\")           // looking for ubuntu based image container\n",
    ".not(has(\"namespace\",\"default\")) // skipping any container present in default namespace\n",
    ".path().by(elementMap())         // converting to Graph output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gremlin introduction\n",
    "\n",
    "Basic gremlin function to play around KubeHound resources. All gremlin function can be access from KubeHound DSL.\n",
    "\n",
    "* `properties()`: get all specified properties for the current element\n",
    "* `values()`: get all specified property values for the current element\n",
    "* `valueMap()` or `elementMap()`: get all specified property values for the current element as a map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Group results by key and value. This allows us to display some important value. \n",
    "\n",
    "* `group()`: group([key]).by(keySelector).by(valueSelector)  \n",
    "* `unfold()`: unfold the incoming list and continue processing each element individually"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "kh.pods()                 // get all the pods\n",
    ".group().by(\"namespace\")  // group by namespaces\n",
    ".by(\"name\")               // filter only the name\n",
    ".unfold()                 // transform the result to a list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Group and Count results by key. This gets metrics and KPI around k8s resources."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "kh.pods()                      // get all the pods\n",
    ".groupCount().by(\"namespace\")  // group and count by namespaces\n",
    ".unfold()                      // transform the result to a list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When using text value you can do some pattern matching using `TextP.<cmd>`\n",
    "    \n",
    "_Note:_ this can slows down a lot the query (not using index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -d name -g class -le 50 -p inv,oute\n",
    "kh.containers()          // get all containers\n",
    "    // retrieve all registry.k8s.io/* image\n",
    ".has(\"image\", TextP.containing(\"registry.k8s.io\"))\n",
    ".path().by(elementMap()) // format it as graph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Classic operator that are useful to scope items of the research:\n",
    "\n",
    "* `limit()`: Limit the number of results \n",
    "* `or()`: Classic `OR` operator, useful when selecting resources by properties\n",
    "* `dedup()`: Will remove any duplicate on the object output (needs to scope to specific properties to make it work)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "kh.containers()  // get all the containers\n",
    ".values(\"image\") // extract the image properties\n",
    ".dedup()         // deduplicate the results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The step-modulator `by()` can be added in addition to other step to modulate the results. It can be added one or multiple times.  The `by()` modulator is commonly used with steps `aggregate()`, `dedup()`, `group()`, `groupCount()`, `order()`, `path()`, `select()`, `tree()`, and more.\n",
    "\n",
    "* `by()`: If a step is able to accept functions, comparators, etc. then by() is the means by which they are added (like group() step)\n",
    "\n",
    "One modulator for the `group()` filter for the `key`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "kh.endpoints()\n",
    ".group()\n",
    ".by(\"port\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second modulator for the `group()` filter for the `value`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "kh.endpoints()\n",
    ".group()\n",
    ".by(\"port\")\n",
    ".by(\"portName\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are some defined value to access specific “properties” of the vertices:\n",
    "\n",
    "* `labels()` or `label`: It takes an Element and extracts its label from it.\n",
    "* `key()` or `key`: It takes a Property and extracts the key from it.\n",
    "* `value()` or `value`: It takes a Property and extracts the value from it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "kh.V()        // get all the vertices\n",
    ".groupCount() // group and count occurencies\n",
    ".by(label)    // count by label of vertices\n",
    ".unfold()     // output as a list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## KubeHound RBAC\n",
    "\n",
    "A permission set is the combination of role and role binding. The reason is that RoleBinding can “downgrade” the scope of a cluster role."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "kh.permissions() // get the permissionsets\n",
    ".valueMap()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "List all the permission set flagged as `critical()`. This is equivalent to `.has(\"critical\",true)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "kh.permissions()                 // get the permissionsets\n",
    ".critical()                      // limit to criticalAsset only\n",
    ".valueMap(\"name\",\"role\",\"rules\") // filter to specific properties"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Intermediate Gremlin\n",
    "\n",
    "When building a path you need to access Edges and Vertices to know when to stop the path.\n",
    "\n",
    "* `outV()`: get all outgoing vertices \n",
    "* `inV()`: get all incoming vertices \n",
    "* `outE()`: get all outgoing edges \n",
    "* `inE()`: get all incoming edges\n",
    "* `out()`: get all adjacent vertices connected by outgoing edges\n",
    "\n",
    "Note: you filter the elements you want to select with labels.\n",
    "\n",
    "Example using `out*()`, building the attacks() DSL function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "\n",
    "kh.containers().outE().inV().path().by(valueMap())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the equivalent to `attacks()`. You should get the same results with it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "\n",
    "kh.containers().attacks().by(elementMap())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To build a path you need to iterate through the element and checks at every step if you want to stop or not.\n",
    "\n",
    "* `loops()`: Indicate the number of iteration\n",
    "* `repeat()`: Define the action you want to iterate\n",
    "* `until()`: Set the condition for the loop\n",
    "* `simplePath()`: Create a path with avoiding cyclic loop that will break the graph\n",
    "\n",
    "To build a path you need to iterate through the element and checks at every step if you want to stop or not.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "\n",
    "kh.endpoints().\n",
    "repeat(\n",
    "  outE().inV().simplePath()  // Building the path\n",
    ").until(\n",
    "    has(\"critical\", true)    // Stop when meeting a critical asset\n",
    "    .or().loops().is(4)      // Stop after 4 iteration\n",
    ").has(\"critical\", true)      // Keep only path ending with a critical asset\n",
    ".path().by(elementMap())     // Output as a graph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is an equivalent to `criticalPaths(4)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "\n",
    "kh.endpoints()    // Start with all endpoints\n",
    ".criticalPaths(4) // Build criticalPath with 4 max hops\n",
    ".by(elementMap()) // Output as a graph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To extract the first element of a path, the local function allows to scope to the first resources.\n",
    "\n",
    "* `local()`: Its purpose is to execute a child traversal on a single element within the stream."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%gremlin -d class -g critical -le 50 -p inv,oute\n",
    "\n",
    "kh.endpoints()    // List all endpoints\n",
    ".criticalPaths()  // Generate the criticalPaths\n",
    ".limit(local,1)   // Extract the first element\n",
    ".dedup()          // Deduplicating result\n",
    ".valueMap()       // Json output of the vertices properties"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
